Synchronization of computer peripheral effects

ABSTRACT

Synchronization of effects across multiple devices using local, distributed control, and eliminating the need for central direction of the effects with the associated large bandwidth required. In one embodiment, each device stores data and instructions for creating its own effects. The devices periodically communicate with a host or each other to maintain synchronization, and compensate for any drift. This approach minimizes the use of bandwidth and battery power in the devices.

BACKGROUND OF THE INVENTION

The present invention relates to the synchronization of lighting andother effects on computer peripheral devices.

Many devices, such as keyboards and mice, have decorative or indicatorlighting. In particular, many input and control devices used for gaminghave such lighting and other effects, such as vibrations. An exampledescribing keyboard lighting effects is US Pub. No. 20150108903,

Logitech and other keyboard and peripheral makers have keyboards andother devices with cosmetic lighting effects. Razer and Corsair makekeyboards with lighting effects that are customizable and can besynchronized with lighting effects on other devices, such as a mouse. Itis believed that such devices use a personal computer as thesynchronization source, with the PC sending out commands to each deviceover a wired USB connection. The PC coordinates the synchronization.

BRIEF SUMMARY OF THE INVENTION

The present invention provides the synchronization of effects acrossmultiple devices using local, distributed control, and eliminating theneed for central direction of the effects with the associated largebandwidth required. In one embodiment, each device stores data andinstructions for creating its own effects. The devices periodicallycommunicate with a host or each other to maintain synchronization, andcompensate for any drift. This approach minimizes the use of bandwidthand battery power in the devices.

In one embodiment, each device initiates an identified effect inresponse to a start signal from a host. The device then periodicallyprovides an indication to the host of where it is in the effectsequence. The host sends a synchronization correction message to thedevice when it is out of synchronization by more than a predeterminedamount. The report can simply be a timer indicating the position in theeffect, which is synchronized with a timer in the host. In oneembodiment, the device report signal is sent only once, in the firsthalf, or near the beginning of an effect cycle, such as ¼ or ⅓ into theeffect.

In one embodiment, the user can select an effect, and select to have itsynchronized with other devices. Once synchronized, the devices staysynchronized without further user input, even if the user changes theeffects.

In one embodiment, the effect is displayed simultaneously on a hostcomputer, and is separately generated on the host computer.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of multiple synchronized devices according toan embodiment of the invention.

FIG. 2 is a timing diagram illustrating the drift and synchronizingbetween a host and multiple devices according to an embodiment of theinvention.

FIG. 3 is a diagram of the messaging for synchronizing between a hostand a device according to an embodiment of the invention.

DETAILED DESCRIPTION OF THE INVENTION

FIG. 1 is a block diagram of multiple synchronized devices according toan embodiment of the invention. A keyboard 102, mouse 104 and personalcomputer 106 are shown. Keyboard 102 has a lighting system 108, andmouse 104 has a lighting system 110. Additional devices with their ownlighting systems, or other effects to be synchronized, can be added. Forexample, a game pad or game controller can also have lighting or othereffects. Other input devices would include a headset, a joystick,touchpad, steering wheel, or any other computer or gaming input device.Alternately, effects systems can be placed in non-peripheral devices,such as a mousepad, a stand, or the frame or display of the host itself.

Keyboard 102 also has a vibrator 112 and a speaker 114 for providinghaptic and sound effects which can be synchronized as well. A controller116 controls the various effects systems using a program in programmemory 118 and data in a data memory 120. In one embodiment, programmemory 118 can be partially in firmware or ROM. A transceiver 122 allowsthe wireless downloading of effects, and the wireless reporting ofeffect timing.

Mouse 104 can similarly have a vibrator 124 and speaker 126. It also hasa controller 128, data memory 130, program memory 132, transceiver 134and antenna 136. The transceiver and antenna also allows for thedownloading of synchronization effects and programs and the reportingback to personal computer 106 or to keyboard 102 of the status of thevarious effects.

Personal computer 106 has its own transceiver 140. Alternately, a routeror hub 142 could be used to interconnect the various devices thatprovide the communication link. In addition, the router or hub or otherdevice could function as the host, as well as providing thecommunication link.

FIG. 2 is a timing diagram illustrating the drift and synchronizingbetween a host and the devices according to an embodiment of theinvention. FIG. 2 shows a host 202, a device A (204) and a device B(206). These could be, for example, the PC, keyboard and mouse ofFIG. 1. When activated, device 204 sends a report 208 to the hostindicating that it has arrived and is active. The host can then send acommand 210 to the device to start the effect on the device(alternately, the device can notify the host when it is starting theeffect). At the same time, the host will independently compute and runthe effect itself (or start a timer for the effect period) so that itcan monitor the appropriate progress in the effect on the devices. Thecomputed effect can also be displayed on the host. Alternately, the hostcan simply run a timer for the effect.

Device 204 then periodically reports the effect status as indicated bycommands 212, 214 and 216. In the example shown, the host determinesthat there is 1% drift at report 212, 2% drift at report 214 and 3%drift at report 216. In this particular example, the host is programmedto reestablish synchronization, and correct the drift, when the driftequals or exceeds 3%. Thus, the host sends a synchronization signal 218to device 204. Device 204 continues to send reports, such as a report220, which may indicate a 1% drift again.

In the example shown, a second device 206 is activated and sends adevice arrival notification 222 to the host. The host then starts thesame effect on both devices at the same time, using command 224 todevice 204 and command 226 to device 206 (alternately, the same commandcan be broadcast to both). This is followed by the continuous reportsfrom devices 204 and 206 as shown, with the host sending asynchronization command to device 204 and device 206 when the reportsindicate that a particular device is 3% or more out of synchronization.

As can be seen in FIG. 2, Device A and Device B are different such thatDevice A tends to drift away from the host at a faster rate than DeviceB. When the device is attached, the host initiates the effect on thedevice and remembers the time (relative to the host) when the effect wasstarted. The device periodically sends an event to indicate the positionof the device effect. The event is sent from the device at around ⅓ ofthe effect period. For example, if the period of the event is 12seconds, the event is sent 4 seconds after the beginning of the effect.The host checks the position of the effect and compares it to its localposition. Once the drift has exceeded a heuristically determinedpercentage (usually based on what is perceptible to the user). a requestis sent to the device to go “backwards” or “forwards” relative to thedrift amount. The figure uses an example of 3% drift. At the end of theeffect, the device slows down or jumps ahead appropriately such that itis closer in sync with the host. When Device B is attached, the effectis restarted on all the devices, and the clock is reset on the host.Synchronization between multiple devices is accomplished by syncing tothe same host.

In more detail, in one embodiment, after the first device is activated,the host reads the effect parameters of the device. The host (LGS) takesthe period of the effect as a reference. The host runs a timerLGSTIMER_REF with the same period. LGS sends SWControl to enable device1. LGS can initialize its own LGSTIMER_REF with the device1 effectperiod counter just received. It will thus avoid large corrections onthe device. At this point LGS and device1 should have both timer effectssynchronized. If the device 1 period counter drifts from LGSTIMER_REF,LGS sends the drift value to device 1. The drift value is calculatedthis way:

drift=Min[ Period- abs(LGSTIMER_REF- DeviceCounter) ,abs(LGSTIMER_REF-DeviceCounter)]the sign of drift is:

negative if Period- abs(LGSTIMER_REF- DeviceCounter) >abs(LGSTIMER_REF-DeviceCounter) positive if Period - abs(LGSTIMER_REF -DeviceCounter) < abs(LGSTIMER_REF -DeviceCounter)

Device 2 can be subsequently activated. LGS reads effect parameters ofdevice 2. LGS makes sure the effect parameters are the same as device 1,if not LGS should write the same settings (at least in terms of effectperiod). LGS sends SWControl enable to device 2. LGS receivesdevice2_period_counter, compares it to LGSTIMER_REF and sends to device2 the drift value, so that device 2 will make the adjustment. Dependingon the drift value, during the first steps the effect on device 2 canstop or speed up for a moment. The drift comparison is repeated on eachdevice.

FIG. 3 is a diagram of the messaging for synchronizing between a hostand a device according to an embodiment of the invention. This diagramis an example of a sawtooth effect started from a host to the device.The host keeps the master clock of the synchronized event. Due to errorson the device, the effect plays longer on the device than the host. Thehost waits for 3% drift before asking the device to synchronize itselfto the host. When the request is sent by the host to the device, thedevice adjusts the effect to finish the period quicker by the driftamount, as shown by abbreviated sawtooth 302. When the next period ofthe device effect starts, it is back in sync with the host effect. Inthe example of FIG. 3, the report from the device is sent near thebeginning of each cycle, or sawtooth, of the effect.

In one embodiment, a user interface is provided on the PC or other hostdevice. The user can select synchronization once, and thereaftersynchronization is maintained for all effects without further input fromthe user. The synchronization is maintained when devices power off andon again, with the effects starting again. The user can change theeffects, such as the type of effect, color, etc., and thesynchronization is carried over and maintained. In one embodiment, theUser Interface allows the user to apply the same effect to all devices.In another embodiment, the user can change the effect on one device. Ifsynchronization has been selected, the new effect will then bepropagated to all the other devices automatically by the host.

Various methods can be used to accomplish the synchronization. Thedevice can simply jump to the timing position the host commands.Alternately, the rate of change of the effect can be sped up, or sloweddown, so there isn't an abrupt change as it comes back intosynchronization. The change can take place in a portion of the effectwhere it will be less noticeable to the user, such as at the end of theeffect cycle, or where it is the least bright.

Various other embodiments can be used. For longer effect cycles,multiple reports can be sent. Alternately, instead of periodicallyreporting, a device can send a single report, such as when ¼ or ⅓ of theeffect has passed. This may be used, for example, for shorter effects.This cuts down on the amount of signaling required, with littlenoticeable difference for shorter effects, or for devices that maintainsynchronization well. In one embodiment, the host can monitor how well adevice stays synchronized over time, and can download a program changeto the device to require reporting more or less often, depending onhistorical performance.

In another embodiment, the host need not have its own time, but canreceive effect start and report signals from the multiple devices,compare them, and send out a correction to one or both. For example, onedevice could be instructed to slow down, and another to speed up, tolessen the abruptness of the change on one device.

In another embodiment, the devices can communicate directly with eachother, rather than with a host. For example, a bigger device, such as akeyboard, may have more processing power than, for example, a mouse. Thekeyboard could then act both as a device and the host. Alternately, thehost could be a smartphone or any other device.

In one embodiment, the program download to the device is done usingWiFi, while the reports and synch commands use Bluetooth. In oneembodiment, the reports and commands are sent as packets interspersedwith transmitted data on a WiFi or other wireless channel, such as sounddata to and from a headset.

A variety of effects can be used. For example, for lighting effects,individual keys or buttons could be lit up with different colors, or thesame color. A wave or other pattern can periodically sweep through thebuttons, or the effects can be provided on or around the keys hit by theuser. The effects could also light up the whole housing, or itsperimeter. The keys could be grouped in zones with the same color oreffect. A glowing effect could simply increase and decrease theintensity. The lighting effects can be coordinated with sound effects,haptic effects or other effects, or those effects could be used insteadof lighting effects.

In an alternate embodiment, the host can periodically send asynchronization command to the device, rather than having the deviceperiodically report. However, the inventors have determined that lesspower is used with device reporting, since the device then needs to beconstantly listening, and each command from the host must be processed,using power, regardless of whether synchronization is needed.

In one embodiment, effects are synchronized even though the effects aredifferent. For example, one device may use RGB (Red, Green, Blue)colors, while another uses CMYK (Cyan, Magenta, Yellow, and Key[black]). The effects may also be different in brightness, or could becompletely different effects that are coordinated in timing to achieve adesired overall effect, like the coordinating of different instrumentsin an orchestra.

In one embodiment, basic or simple effects are embedded in firmware orROM. More complicated effects are programmed in software and downloadedto the device, and can operate independently of the firmware effects, orcan utilize the firmware effects while adding to them. Alternately, morecomplex effects can be imbedded in higher value products. In oneembodiment, effects are programmed in firmware, with parameters that canbe downloaded to vary the effect.

As will be understood by those of skill in the art, the presentinvention can be embodied in other forms. Accordingly, the foregoingdescription is illustrating, but not limiting, of the scope of theinvention which is set forth in the following claims.

What is claimed is:
 1. A system for synchronizing effects among multipledevices, comprising: a first device; a first effect system in the firstdevice programmed to provide a first effect over a period of time and toprovide a series of first timing reports on the current timing of thefirst effect; a transceiver coupled to the first device and configuredto receive the first timing reports from the first effect system andtransmit the reports wirelessly; the first effect subsystem beingprogrammed to receive a first synchronization signal and to modify thetiming of the first effect in accordance with the first synchronizationsignal; non-transitory computer readable code for execution on a host,having instructions to process a signal indicating the synchronizedstart of the first effect and a second effect; initiate a host timercorresponding to the synchronized start of the first and second effects;receive the first report and a second report and compare them to thehost timer; determine if the timing of the first effect has varied fromthe host timer by more than a predetermined amount; send the firstsynchronization signal to the first device if the first effect hasvaried from the host timer by more than the predetermined amount;determine if the timing of a second effect has varied from the hosttimer by more than a predetermined amount; and send the secondsynchronization signal to a second device if the second effect hasvaried from the host timer by more than the predetermined amount.
 2. Thesystem of claim 1 further comprising: a second device; a second effectsystem in said second device programmed to provide the second effectover a period of time and to provide a series of second reports on thecurrent timing of the second effect; a second transceiver coupled to thesecond device and configured to receive the second reports and transmitthe reports wirelessly; the second effect subsystem being programmed toreceive a second synchronization signal and to modify the timing of thesecond effect in accordance with the second synchronization signal; 3.The system of claim 1 wherein: the host transmits to the first andsecond devices a signal indicating the synchronized start of the firstand second effects.
 4. The system of claim 1 wherein the first andsecond devices send the first and second reports once in each cycle ofthe first and second effects, in the first half of the effect cycle. 5.The system of claim 1 wherein the reports include a device timerindicating the position in the effect, and wherein the host includes ahost timer tracking the position of the effect.
 6. The system of claim 1further comprising, on the host: a user interface allowing a user tochange the effects; and code for maintaining the devices synchronizedwithout further user input after changes to the effects.
 7. A system forsynchronizing effects among multiple devices, comprising: a firstdevice; a first effect system in said first device programmed to providea first effect over a period of time and to provide a series of firstreports on the current timing of the first effect; a transceiver coupledto the first device and configured to receive the first reports andtransmit the reports wirelessly; the first effect subsystem beingprogrammed to receive a first synchronization signal and to modify thetiming of the first effect in accordance with the first synchronizationsignal; non-transitory computer readable code for execution on a host,having instructions to transmit to the first and second devices a signalindicating the synchronized start of the first and second; initiate ahost timer corresponding to the synchronized start of the first andsecond effects; receive the first report and a second report and comparethem to the host timer; determine if the timing of the first effect hasvaried from the host timer by more than a predetermined amount; send thefirst synchronization signal to the first device if the first effect hasvaried from the host timer by more than the predetermined amount;determine if the timing of a second effect has varied from the hosttimer by more than a predetermined amount; send the secondsynchronization signal to a second device if the second effect hasvaried from the host timer by more than the predetermined amount;provide a user interface allowing a user to change the effects; maintainthe devices synchronized without further user input after changes to theeffects wherein the first and second devices send the first and secondreports once in each cycle of the first and second effects, in the firsthalf of the effect cycle; wherein the reports include a device timerindicating the position in the effect, and wherein the host includes ahost timer tracking the position of the effect; and wherein the effectsare one or more of lighting, sound and haptic effects.
 8. A method forsynchronizing effects among multiple devices, comprising: providing afirst effect in a first device over a period of time and to provide aseries of first reports on the current timing of the first effect;transmitting the reports wirelessly; receiving a first synchronizationsignal and modifying the timing of the first effect in accordance withthe first synchronization signal; processing, at a host, a signalindicating the synchronized start of the first effect and a secondeffect; initiating a host timer corresponding to the synchronized startof the first and second effects; receiving at the host the first reportand a second report and comparing them to the host timer; determine ifthe timing of the first effect has varied from the host timer by morethan a predetermined amount; sending the first synchronization signal tothe first device if the first effect has varied from the host timer bymore than the predetermined amount; determining if the timing of asecond effect has varied from the host timer by more than apredetermined amount; and sending the second synchronization signal to asecond device if the second effect has varied from the host timer bymore than the predetermined amount.
 9. The method of claim 8 wherein:transmitting by the host to the first and second devices a signalindicating the synchronized start of the first and second effects. 10.The method of claim 8 further comprising: sending the first and secondreports once in each cycle of the first and second effects, in the firsthalf of the effect cycle.
 11. The method of claim 8 further comprising:indicating with a device timer the position in the effect, and trackingthe position of the effect with a host timer.
 12. The method of claim 8further comprising: providing a user interface on the host allowing auser to change the effects; and maintaining the devices synchronizedwithout further user input after changes to the effects.
 13. Computerreadable code on at least two non-transitory computer readable media forsynchronizing effects among multiple devices, comprising: providing afirst effect in a first device over a period of time and to provide aseries of first reports on the current timing of the first effect;transmitting the reports wirelessly; receiving a first synchronizationsignal and modifying the timing of the first effect in accordance withthe first synchronization signal; processing, at a host, a signalindicating the synchronized start of the first effect and a secondeffect; initiating a host timer corresponding to the synchronized startof the first and second effects; receiving at the host the first reportand a second report and comparing them to the host timer; determine ifthe timing of the first effect has varied from the host timer by morethan a predetermined amount; sending the first synchronization signal tothe first device if the first effect has varied from the host timer bymore than the predetermined amount; determining if the timing of asecond effect has varied from the host timer by more than apredetermined amount; and sending the second synchronization signal to asecond device if the second effect has varied from the host timer bymore than the predetermined amount.
 14. The computer readable code ofclaim 13 further comprising non-transitory computer readable media for:transmitting by the host to the first and second devices a signalindicating the synchronized start of the first and second effects. 15.The computer readable code of claim 13 further comprising non-transitorycomputer readable media for: sending the first and second reports oncein each cycle of the first and second effects, in the first half of theeffect cycle.
 16. The computer readable code of claim 13 furthercomprising non-transitory computer readable media for: indicating with adevice timer the position in the effect, and tracking the position ofthe effect with a host timer.
 17. The computer readable code of claim 13further comprising non-transitory computer readable media for: providinga user interface on the host allowing a user to change the effects; andmaintaining the devices synchronized without further user input afterchanges to the effects.
 18. The computer readable code of claim 13:wherein the effects are one or more of lighting, sound and hapticeffects.